Visualizing machine learning-based predictions of postpartum depression risk for lay audiences.

OBJECTIVES: To determine if different formats for conveying machine learning (ML)-derived postpartum depression risks impact patient classification of recommended actions (primary outcome) and intention to seek care, perceived risk, trust, and preferences (secondary outcomes). MATERIALS AND METHODS: We recruited English-speaking females of childbearing age (18-45 years) using an online survey platform. We created 2 exposure variables (presentation format and risk severity), each with 4 levels, manipulated within-subject. Presentation formats consisted of text only, numeric only, gradient number line, and segmented number line. For each format viewed, participants answered questions regarding each outcome. RESULTS: Five hundred four participants (mean age 31 years) completed the survey. For the risk classification question, performance was high (93%) with no significant differences between presentation formats. There were main effects of risk level (all P < .001) such that participants perceived higher risk, were more likely to agree to treatment, and more trusting in their obstetrics team as the risk level increased, but we found inconsistencies in which presentation format corresponded to the highest perceived risk, trust, or behavioral intention. The gradient number line was the most preferred format (43%). DISCUSSION AND CONCLUSION: All formats resulted high accuracy related to the classification outcome (primary), but there were nuanced differences in risk perceptions, behavioral intentions, and trust. Investigators should choose health data visualizations based on the primary goal they want lay audiences to accomplish with the ML risk score.

Do you want to promote recall, perceptions, or behavior? The best data visualization depends on the communication goal.

Data visualizations can be effective and inclusive means for helping people understand health-related data. Yet numerous high-quality studies comparing data visualizations have yielded relatively little practical design guidance because of a lack of clarity about what communicators want their audience to accomplish. When conducting rigorous evaluations of communication (eg, applying the ISO 9186 method), describing the process simply as evaluating "comprehension" or "interpretation" of visualizations fails to do justice to the true range of outcomes being studied. We present newly developed taxonomies of outcome measures and tasks that are guiding a large-scale systematic review of the health numbers communication literature. Using these taxonomies allows a designer to determine whether a specific data presentation format or feature supports or inhibits the desired audience cognitions, feelings, or behaviors. We argue that taking a granular, outcomes-based approach to designing and evaluating information visualization research is essential to deriving practical, actionable knowledge from it.

Development of a novel instrument to characterize telemedicine programs in primary care.

BACKGROUND: Given the rapid deployment of telemedicine at the onset of the COVID - 19 pandemic, updated assessment methods are needed to study and characterize telemedicine programs. We developed a novel semi - structured survey instrument to systematically describe the characteristics and implementation processes of telemedicine programs in primary care. METHODS: In the context of a larger study aiming to describe telemedicine programs in primary care, a survey was developed in 3 iterative steps: 1) literature review to obtain a list of telemedicine features, facilitators, and barriers; 2) application of three evaluation frameworks; and 3) stakeholder engagement through a 2-stage feedback process. During survey refinement, items were tested against the evaluation frameworks while ensuring it could be completed within 20-25 min. Data reduction techniques were applied to explore opportunity for condensed variables/items. RESULTS: Sixty initially identified telemedicine features were reduced to 32 items / questions after stakeholder feedback. Per the life cycle framework, respondents are asked to report a month in which their telemedicine program reached a steady state, i.e., "maturation". Subsequent questions on telemedicine features are then stratified by telemedicine services offered at the pandemic onset and the reported point of maturation. Several open - ended questions allow for additional telemedicine experiences to be captured. Data reduction techniques revealed no indication for data reduction. CONCLUSION: This 32-item semi-structured survey standardizes the description of primary care telemedicine programs in terms of features as well as maturation process. This tool will facilitate evaluation of and comparisons between telemedicine programs across the United States, particularly those that were deployed at the pandemic onset.

Women's perspectives on the use of artificial intelligence (AI)-based technologies in mental healthcare.

This study aimed to evaluate women's attitudes towards artificial intelligence (AI)-based technologies used in mental health care. We conducted a cross-sectional, online survey of U.S. adults reporting female sex at birth focused on bioethical considerations for AI-based technologies in mental healthcare, stratifying by previous pregnancy. Survey respondents (n = 258) were open to AI-based technologies in mental healthcare but concerned about medical harm and inappropriate data sharing. They held clinicians, developers, healthcare systems, and the government responsible for harm. Most reported it was "very important" for them to understand AI output. More previously pregnant respondents reported being told AI played a small role in mental healthcare was "very important" versus those not previously pregnant (P = .03). We conclude that protections against harm, transparency around data use, preservation of the patient-clinician relationship, and patient comprehension of AI predictions may facilitate trust in AI-based technologies for mental healthcare among women.

The Telemedicine Experience in Primary Care Practices in the United States: Insights From Practice Leaders.

PURPOSE: The need to rapidly implement telemedicine in primary care during the coronavirus disease 2019 (COVID-19) pandemic was addressed differently by various practices. Using qualitative data from semistructured interviews with primary care practice leaders, we aimed to report commonly shared experiences and unique perspectives regarding telemedicine implementation and evolution/maturation since March 2020. METHODS: We administered a semistructured, 25-minute, virtual interview with 25 primary care practice leaders from 2 health systems in 2 states (New York and Florida) included in PCORnet, the Patient-Centered Outcomes Research Institute clinical research network. Questions were guided by 3 frameworks (health information technology evaluation, access to care, and health information technology life cycle) and involved practice leaders' perspectives on the process of telemedicine implementation in their practice, with a specific focus on the process of maturation and facilitators/barriers. Two researchers conducted inductive coding of qualitative data open-ended questions to identify common themes. Transcripts were electronically generated by virtual platform software. RESULTS: Twenty-five interviews were administered for practice leaders representing 87 primary care practices in 2 states. We identified the following 4 major themes: (1) the ease of telemedicine adoption depended on both patients' and clinicians' prior experience using virtual health platforms, (2) regulation of telemedicine varied across states and differentially affected the rollout processes, (3) visit triage rules were unclear, and (4) there were positive and negative effects of telemedicine on clinicians and patients. CONCLUSIONS: Practice leaders identified several challenges to telemedicine implementation and highlighted 2 areas, including telemedicine visit triage guidelines and telemedicine-specific staffing and scheduling protocols, for improvement.

Limited English Proficiency in Older Adults Referred to the Cardiovascular Team.

Limited English proficiency (LEP) is defined as individuals in whom English is not the primary language and who have limited ability to read, speak, write, or understand the English language. Cardiovascular (CV) team members routinely encounter language barriers in their practice. These barriers have a significant impact on the quality of CV care that patients with LEP receive. Despite evidence demonstrating the negative association between language barriers and health disparities, the impact on CV care is insufficiently known. In addition, older adults with CV disease and LEP are facing increasing risk of adverse events when complex medical information is not optimally delivered. Overcoming language barriers in CV care will need a thoughtful approach. Although well recognized, the initial step will be to continue to highlight the importance of language needs identification and appropriate use of professional interpreter services. In parallel, a health system-level approach is essential that describes initiatives and key policies to ensure a high-level quality of care for a growing LEP population. This review aims to present the topic of LEP during the CV care of older adults, for continued awareness along with practical considerations for clinical use and directions for future research.

Prediction of left ventricular ejection fraction changes in heart failure patients using machine learning and electronic health records: a multi-site study.

Left ventricular ejection fraction (EF) is a key measure in the diagnosis and treatment of heart failure (HF) and many patients experience changes in EF overtime. Large-scale analysis of longitudinal changes in EF using electronic health records (EHRs) is limited. In a multi-site retrospective study using EHR data from three academic medical centers, we investigated longitudinal changes in EF measurements in patients diagnosed with HF. We observed significant variations in baseline characteristics and longitudinal EF change behavior of the HF cohorts from a previous study that is based on HF registry data. Data gathered from this longitudinal study were used to develop multiple machine learning models to predict changes in ejection fraction measurements in HF patients. Across all three sites, we observed higher performance in predicting EF increase over a 1-year duration, with similarly higher performance predicting an EF increase of 30% from baseline compared to lower percentage increases. In predicting EF decrease we found moderate to high performance with low confidence for various models. Among various machine learning models, XGBoost was the best performing model for predicting EF changes. Across the three sites, the XGBoost model had an F1-score of 87.2, 89.9, and 88.6 and AUC of 0.83, 0.87, and 0.90 in predicting a 30% increase in EF, and had an F1-score of 95.0, 90.6, 90.1 and AUC of 0.54, 0.56, 0.68 in predicting a 30% decrease in EF. Among features that contribute to predicting EF changes, baseline ejection fraction measurement, age, gender, and heart diseases were found to be statistically significant.

Recentering responsible and explainable artificial intelligence research on patients: implications in perinatal psychiatry.

In the setting of underdiagnosed and undertreated perinatal depression (PD), Artificial intelligence (AI) solutions are poised to help predict and treat PD. In the near future, perinatal patients may interact with AI during clinical decision-making, in their patient portals, or through AI-powered chatbots delivering psychotherapy. The increase in potential AI applications has led to discussions regarding responsible AI and explainable AI (XAI). Current discussions of RAI, however, are limited in their consideration of the patient as an active participant with AI. Therefore, we propose a patient-centered, rather than a patient-adjacent, approach to RAI and XAI, that identifies autonomy, beneficence, justice, trust, privacy, and transparency as core concepts to uphold for health professionals and patients. We present empirical evidence that these principles are strongly valued by patients. We further suggest possible design solutions that uphold these principles and acknowledge the pressing need for further research about practical applications to uphold these principles.

Community Health Recommendations Driven by mHealth Population Surveillance Data Amongst Burmese Displaced People in Eastern India: A Pilot Usability Assessment of a Mobile Health Application for Data Collection.

With recent increases in armed conflict and forced migration, refugee health has become a growing priority amongst those who work in global health. Refugees and forced migrants, also known as displaced persons, face barriers to accessing health services and are often at an increased risk for adverse health outcomes, such as sexual violence, infectious diseases, poor maternal outcomes, and mental health concerns. Mobile health (mHealth) applications have been shown to increase access and improve health outcomes among refugee populations. Our study aims to evaluate the feasibility of using a novel mHealth application to conduct population health surveillance data collection amongst a population of Myanmar citizens who have been forced to relocate to eastern India. The data collected in a low-resource setting through the mHealth application will be used to identify priority areas for intervention which will assist in the development of a tailored intervention plan that best suits our population.

Designing Support to help Health Communication Professionals Convey Numbers Clearly to the Public - A Needs Assessment and Formative Usability Evaluation.

Communicating health-related probabilities to patients and the public presents challenges, although multiple studies have demonstrated that we can promote comprehension and appropriate application of numbers by matching presentation formats (e.g., percentage, bar charts, icon arrays) to communication goal (e.g., improving recall, decreasing worry, taking action). We used this literature to create goal-driven, evidence-based guidance to support health communicators in conveying probabilities. We then conducted semi-structured interviews with 39 health communicators to understand: communicators' goals for expressing probabilities, formats they choose to convey probabilities, and perceptions of prototypes of our "communicating numbers clearly" guidance. We found that communicators struggled to articulate granular goals for their communication, impeding their ability to select appropriate guidance. Future work should consider how best to support health communicators in selecting granular, differentiable goals to support broadly comprehensible information design.

Identifying Nonpatient Authors of Patient Portal Secure Messages in Oncology: A Proof-of-Concept Demonstration of Natural Language Processing Methods.

PURPOSE: Patient portal secure messages are not always authored by the patient account holder. Understanding who authored the message is particularly important in an oncology setting where symptom reporting is crucial to patient treatment. Natural language processing has the potential to detect messages not authored by the patient automatically. METHODS: Patient portal secure messages from the Memorial Sloan Kettering Cancer Center were retrieved and manually annotated as a predicted unregistered proxy (ie, not written by the patient) or a presumed patient. After randomly splitting the annotated messages into training and test sets in a 70:30 ratio, a bag-of-words approach was used to extract features and then a Least Absolute Shrinkage and Selection Operator (LASSO) model was trained and used for classification. RESULTS: Portal secure messages (n = 2,000) were randomly selected from unique patient accounts and manually annotated. We excluded 335 messages from the data set as the annotators could not determine if they were written by a patient or proxy. Using the remaining 1,665 messages, a LASSO model was developed that achieved an area under the curve of 0.932 and an area under the precision recall curve of 0.748. The sensitivity and specificity related to classifying true-positive cases (predicted unregistered proxy-authored messages) and true negatives (presumed patient-authored messages) were 0.681 and 0.960, respectively. CONCLUSION: Our work demonstrates the feasibility of using unstructured, heterogenous patient portal secure messages to determine portal secure message authorship. Identifying patient authorship in real time can improve patient portal account security and can be used to improve the quality of the information extracted from the patient portal, such as patient-reported outcomes.

Developing Population Health Surveillance Using mHealth in Low-Resource Settings: Qualitative Assessment and Pilot Evaluation.

BACKGROUND: Population surveillance data are essential for understanding population needs and evaluating health programs. Governmental and nongovernmental organizations in western Myanmar did not previous have means for conducting robust, electronic population health surveillance. OBJECTIVE: This study involved developing mobile health (mHealth)-based population health surveillance in a rural, low-resource setting with minimal cellular infrastructure in western Myanmar. This was an early formative study in which our goal was to establish the initial feasibility of conducting mHealth population health surveillance, optimizing procedures, and building capacity for future work. METHODS: We used an iterative design process to develop mHealth-based population health surveillance focused on general demographics (eg, total census, age category, sex, births, and deaths). Interviews were conducted with international consultants (nurse midwives) and local clinicians (nurses and physicians) in Myanmar. Our analytic approach was informed by the Systems Engineering Initiative for Patient Safety work systems model to capture the multilevel user needs for developing health interventions, which was used to create a prototype data collection tool. The prototype was then pilot-tested in 33 villages to establish an initial proof of concept. RESULTS: We conducted 7 interviews with 5 participants who provided feedback regarding the domains of the work system, including environmental, organizational, sociocultural, technological, informational, and task- and people-based considerations, for adapting an mHealth tool. Environmental considerations included managing limited electricity and internet service. Organizational needs involved developing agreements to work within existing government infrastructure as well as leveraging the communal nature of societies to describe the importance of surveillance data collection and gain buy-in. Linguistic diversity and lack of experience with technology were both cited as people- and technology-based aspects to inform prototype design. The use of mobile tools was also viewed as a means to improve the quality of the data collected and as a feasible option for working in settings with limited internet access. Following the prototype design based on the findings of initial interviews, the mHealth tool was piloted in 33 villages, allowing our team to collect census data from 11,945 people for an initial proof of concept. We also detected areas of potentially missing data, which will need to be further investigated and mitigated in future studies. CONCLUSIONS: Previous studies have not focused heavily on the early stages of developing population health surveillance capacity in low- and middle-income countries. Findings related to key design considerations using a work systems lens may be informative to others developing technology-based solutions in extremely low-resource settings. Future work will involve collecting additional health-related data and further evaluating the quality of the data collected. Our team established an initial proof of concept for using an mHealth tool to collect census-related information in a low-resource, extremely rural, and low-literacy environment.

Design and validation of a FHIR-based EHR-driven phenotyping toolbox.

OBJECTIVES: To develop and validate a standards-based phenotyping tool to author electronic health record (EHR)-based phenotype definitions and demonstrate execution of the definitions against heterogeneous clinical research data platforms. MATERIALS AND METHODS: We developed an open-source, standards-compliant phenotyping tool known as the PhEMA Workbench that enables a phenotype representation using the Fast Healthcare Interoperability Resources (FHIR) and Clinical Quality Language (CQL) standards. We then demonstrated how this tool can be used to conduct EHR-based phenotyping, including phenotype authoring, execution, and validation. We validated the performance of the tool by executing a thrombotic event phenotype definition at 3 sites, Mayo Clinic (MC), Northwestern Medicine (NM), and Weill Cornell Medicine (WCM), and used manual review to determine precision and recall. RESULTS: An initial version of the PhEMA Workbench has been released, which supports phenotype authoring, execution, and publishing to a shared phenotype definition repository. The resulting thrombotic event phenotype definition consisted of 11 CQL statements, and 24 value sets containing a total of 834 codes. Technical validation showed satisfactory performance (both NM and MC had 100% precision and recall and WCM had a precision of 95% and a recall of 84%). CONCLUSIONS: We demonstrate that the PhEMA Workbench can facilitate EHR-driven phenotype definition, execution, and phenotype sharing in heterogeneous clinical research data environments. A phenotype definition that integrates with existing standards-compliant systems, and the use of a formal representation facilitates automation and can decrease potential for human error.

Strategies to optimize comprehension of numerical medication instructions: A systematic review and concept map.

OBJECTIVE: To develop evidence-based recommendations for improving comprehension of quantitative medication instructions. METHODS: This review included a literature search from inception to November 2021. Studies were included for the following: 1) original research; 2) compared multiple formats for presenting quantitative medication information on dose, frequency, and/or time; 3) included patients/lay-people; 4) assessed comprehension-related outcomes quantitatively. To classify the studies, we developed a concept map. We weighed 3 factors (risk of bias in individual studies, consistency of findings among studies, and homogeneity of the interventions tested) to generate 3 levels of recommendations. RESULTS: Twenty-one studies were included. Level 1 recommendations are: 1) use visualizations of medication doses for liquid medications, and 2) express instructions in time-periods rather than times per day. Level 2 recommendations include: validate icons, use panels or tables with explanatory text, use visualizations for non-English speaking populations and for those with low health literacy and limited English proficiency. CONCLUSIONS: Visualized liquid medication doses and time period-based administration instructions improve comprehension of numerical medication instructions. Use of visualizations for those with limited health literacy and English proficiency could result in improved outcomes. PRACTICE IMPLICATIONS: Practitioners should use visualizations for liquid medication instructions and time period-based instructions to improve outcomes.

Taxonomies for synthesizing the evidence on communicating numbers in health: Goals, format, and structure.

Many people, especially those with low numeracy, are known to have difficulty interpreting and applying quantitative information to health decisions. These difficulties have resulted in a rich body of research about better ways to communicate numbers. Synthesizing this body of research into evidence-based guidance, however, is complicated by inconsistencies in research terminology and researcher goals. In this article, we introduce three taxonomies intended to systematize terminology in the literature, derived from an ongoing systematic literature review. The first taxonomy provides a systematic nomenclature for the outcome measures assessed in the studies, including perceptions, decisions, and actions. The second taxonomy is a nomenclature for the data formats assessed, including numbers (and different formats for numbers) and graphics. The third taxonomy describes the quantitative concepts being conveyed, from the simplest (a single value at a single point in time) to more complex ones (including a risk-benefit trade-off and a trend over time). Finally, we demonstrate how these three taxonomies can be used to resolve ambiguities and apparent contradictions in the literature.

Social Determinants of Health and Patient Safety: An Analysis of Patient Safety Event Reports Related to Limited English-Proficient Patients.

BACKGROUND: Racial and ethnic disparities in healthcare safety have persisted for decades, particularly for patients with language barriers. Previous studies have investigated the frequency and nature of safety events impacting patients with language barriers; others have proposed solutions to fix them. A gap analysis, however, of how we are currently addressing safety issues and why these efforts have not been effective is lacking. METHOD: This analysis uses reports from a patient safety event reporting system. Reports contain information regarding no-harm (near miss) events and events where harm may have reached the patient. Reports occurring with patients with a preferred language other than English were extracted and analyzed to determine whether the language barrier contributed to the safety event, the language barrier was mentioned in the resolution, and themes were mentioned for addressing language barriers. RESULTS: A subset of 1553 events pertaining to non-English-speaking patients were first categorized as "likely" (3%), "plausibly" (10%), or "unlikely" (87%) related to the patient's language barrier. Second, events related to the patient's language barrier were categorized as directly addressing (19%), indirectly addressing (3%), not mentioning (69%) the language barrier, or containing insufficient information to determine whether the language barrier was addressed (7%). Third, thematic analysis revealed that the most common methods for addressing language barriers included presenting issues to interpreter services and subsequent use of interpreter services. CONCLUSIONS: This study found that it is challenging to determine the direct role of certain social determinants of health (e.g., language barriers) in safety events. In many cases, the language barrier was not addressed in the event report. Furthermore, when the language barrier was addressed, solution themes typically involved weaker, less sustainable suggested actions.

The active role of interpreters in medical discourse - An observational study in emergency medicine.

OBJECTIVE: To study communicative tasks executed and related strategies used by patients, health professionals, and medical interpreters. METHODS: English proficient and limited English proficient emergency department patients were observed. The content of patient-hospital staff communication was documented via pen and paper. Key themes and differences across interpreter types were established through qualitative analysis. Themes and differences across interpreter type were vetted and updated through member checking interviews. RESULTS: 6 English proficient and 9 limited English proficient patients were observed. Key themes in communicative tasks included: establishing, maintaining, updating, and repairing understanding and rapport. All tasks were observed with English proficient and limited English proficient patients. The difference with limited English proficient patients was that medical interpreters played an active role in completing communicative tasks. Telephone-based interpreters faced challenges in facilitating communicative tasks based on thematic comparisons with in-person interpreters, including issues hearing and lost information due to the lack of visual cues. CONCLUSIONS: Professional interpreters play an important role in communication between language discordant patients and health professionals that goes beyond verbatim translation. PRACTICAL IMPLICATIONS: Training for interpreters and health professionals, and the design of tools for facilitating language discordant communication, should consider the role of interpreters beyond verbatim translation.